Sparse multi-class prediction based on the Group Lasso in multinomial logistic regression

2 3 Preface Many classification procedures are based on variable selection methodologies. This master thesis concentrates on continuous variable selection procedures based on the shrinkage principle. Generally, we would like to find sparse prediction rules for multi-class classification problems such that in increases the prediction accuracy but also the interpretability of the obtained prediction rules. For these reasons we have chosen for the multinomial logistic regression model as its penalization procedures perform continuous variable selection and generally lead to sparse prediction rules. Next to the multinomial logistic regression model we have also implemented the Ridge penalization, Lasso penalization, Elastic net penalization, and the Group Lasso. The emphasis of this research lies on the Lasso and Group Lasso. The Lasso performs in a multi-class classification problem a variable selection on individual regression coefficients. In the multinomial regression model each predictor has a regression coefficient per class. The selection of the individual regression coefficients is less logical than the selection of an entire predictor. For this reason it could select redundant predictors leading to more retained predictors and less interpretable prediction rules. To overcome this problem we have developed a Group Lasso procedure with a novel group structure. The advantage of using the Group Lasso is that it performs variable selection on the predefined groups. In our model we developed a group structure which groups all the regression coefficients, i.e. of each class, of each predictor. This group structure facilitates the selection of an entire predictor. We demonstrate on the basis of gene expression profiles of 531 well-characterized Acute Myeloid Leukemia patients that the Group Lasso obtains less predictors with a similar prediction accuracy when compared to the regular Lasso. Finally, the Group Lasso facilitates the comparison of regression coefficients between classes for each predictor, which is not possible with the Lasso. Acknowledgement I would express my appreciation and gratitude to Jelle Goeman, who has been my mentor and supervisor during this study. I would like to thank him for giving me the opportunity to work at such a gentile and successful department. He has the uncanny ability to remain critical and explain difficult problems in way that I could understand them. I look forward to the many projects we could work on together. Next, I would like to express my gratitude to Marcel Reinders, my supervisor during the Master course. Marcel has the great ability to translate problems and descriptions, stemming …